Liver fibrosis recognition method based on medical images and computing device using thereof

ABSTRACT

A liver fibrosis recognition method based on medical images and a computing device using thereof obtains a plurality of first binary images by segmenting a region of interest in each of a plurality of medical images of a liver. A rectangular region is created for each first binary image, and a plurality of second binary images is obtained by generating a second binary according to each rectangular region and the first binary image. A feature map is obtained from each liver medical image and images are generated according to the second binary images and corresponding to the plurality of feature maps. A model for recognition is iteratively trained based on the plurality of final images and recognition of liver fibrosis in patients is then achievable using the model.

FIELD

The present disclosure relates to a technical field of digital medicaltechnology, specifically a liver fibrosis recognition method based onmedical images and a computing device using thereof.

BACKGROUND

Liver fibrosis is a major health threat with high prevalence. Withouttimely diagnosis and treatment, liver fibrosis can develop into livercirrhosis and even hepatocellular carcinoma. While histopathologyremains the gold standard, non-invasive approaches minimize patientdiscomfort and danger. Elastography is a useful non-invasive approach,but it is not always available or affordable, and it can be confoundedby inflammation, presence of liver steatosis, and the patient'setiology. Assessment using conventional ultrasound may be potentiallymore versatile; however, it is a subjective measurement that can sufferfrom insufficient sensitivities, specificities, and high inter- andintra-rater variability. There is impetus for an automated and lesssubjective assessment of liver fibrosis.

Although a relatively understudied topic, related work has advancedautomated Ultrasound fibrosis assessment. In terms of deep convolutionalneural networks (CNNs), Meng et al., put forward a straightforward liverfibrosis parenchyma VGG-16-based classifier, and tested on a smalldataset of 279 images. Only image-wise predictions were performed and nostudy-wise prediction was reported. On the other hand, Liu et al.,correctly identified the value of fusing features from all ultrasoundimages in a study when making a prediction. However, their algorithmrequires exactly 10 images. Real patient studies may contain anarbitrary number of ultrasound scans. Their concatenation of featuresapproach also drastically increases computational and memory costs asmore images are incorporated. Moreover, 13 manually labeled indicatorsare relied on as ancillary supervision, which are typically notavailable without considerable labor costs.

A solution for liver fibrosis recognition is required.

SUMMARY

A first aspect of an embodiment of the present disclosure provides aliver fibrosis recognition method based on medical images. The methodincludes: obtaining a plurality of first binary images by segmenting aregion of interest (interest region) in each of a plurality of livermedical images; creating a rectangular region for each first binaryimage, and obtaining a plurality of second binary images by generating asecond binary image according to each rectangular region and thecorresponding first binary image; extracting a feature map of each livermedical image to obtain a plurality of feature maps; generating aplurality of input images according to the plurality of second binaryimages and corresponding to the plurality of feature maps; iterativelytraining a liver fibrosis recognition model based on the plurality ofinput images; and obtaining a liver fibrosis recognition result by usingthe liver fibrosis recognition model to recognize a liver medical imageto be recognized.

A second aspect of an embodiment of the present disclosure provides acomputing device, which includes: at least one processor; and a storagedevice storing one or more programs which when executed by the at leastone processor, cause the at least one processor to: segment an interestregion in each of a plurality of liver medical images and obtain aplurality of first binary images; create a rectangular region for eachfirst binary image, and obtain a plurality of second binary images bygenerating a second binary image according to each rectangular regionand the corresponding first binary image; extract a feature map of eachliver medical image to obtain a plurality of feature maps; generate aplurality of input images according to the plurality of second binaryimages and corresponding to the plurality of feature maps; iterativelytrain a liver fibrosis recognition model based on the plurality of inputimages; and obtain a liver fibrosis recognition result by using theliver fibrosis recognition model to recognize a liver medical image tobe recognized.

A third aspect of an embodiment of the present disclosure provides anon-transitory storage medium having stored thereon instructions that,when executed by a processor of a computing device, causes the computingdevice to perform a liver fibrosis recognition method, the methodincludes: segmenting an interest region in each of a plurality of livermedical images and obtaining a plurality of first binary images;creating a rectangular region for each first binary image, and obtaininga plurality of second binary images by generating a second binary imageaccording to each rectangular region and the corresponding first binaryimage; extracting a feature map of each liver medical image to obtain aplurality of feature maps; generating a plurality of input imagesaccording to the plurality of second binary images and corresponding tothe plurality of feature maps; iteratively training a liver fibrosisrecognition model based on the plurality of input images; and obtaininga liver fibrosis recognition result by using the liver fibrosisrecognition model to recognize a liver medical image to be recognized.

In the embodiments of the present disclosure, by avoiding a CNNoverfitting on non-relevant image features (spurious or backgroundfeatures), the network is forced to focus on a clinical region ofinterest (ROI), encompassing the liver parenchyma and upper border. Aglobal hetero-image fusion (GHIF) is introduced, which allows the CNN tofuse features from any arbitrary number of images in an Ultrasoundstudy, increasing its versatility and flexibility. Finally, a“style”-based view-specific paranrieterization (VSP) is used to tailorthe CNN processing based on the particular view of each ultrasound imagebased on 6 common liver ultrasound views, while keeping the majority ofparameters the same across views. The result is a highly robust andpractical liver fibrosis assessment solution.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows a schematic flow chart of an embodiment of a liver fibrosisrecognition method based on medical images according to the presentdisclosure.

FIG. 2 shows a schematic structural diagram of a network architecturefor training a liver fibrosis recognition model according to the presentdisclosure.

FIG. 3 shows a schematic diagram of an example of a region of interestand a rectangular box according to the present disclosure.

FIG. 4 shows a schematic diagram of an example of liver Ultrasoundimages with different image views according to the present disclosure.

FIG. 5 shows a schematic structural diagram of an embodiment of a liverfibrosis recognition device based on medical images according to thepresent disclosure.

FIG. 6 shows a schematic structural diagram of a computing deviceapplying the method of FIG. 1 according to the present disclosure.

DETAILED DESCRIPTION

The embodiments of the present disclosure are described with referenceto the accompanying drawings. Described embodiments are merelyembodiments which are a part of the present disclosure, and do notinclude every embodiment. All other embodiments obtained by those ofordinary skill in the art based on the embodiments of the presentdisclosure without creative efforts are within the scope of the claims.

Terms such as “first”, “second” and the like in the specification and inthe claims of the present disclosure and the above drawings are used todistinguish different objects, and are not intended to describe aspecific order. Moreover, the term “include” and any variations of theterm “include” are intended to indicate a non-exclusive inclusion. Forexample, a process, a method, a system, a product, or a device whichincludes a series of steps or units is not limited to steps or unitswhich are listed, but can include steps or units which are not listed,or can include other steps or units inherent to such processes, methods,products, and equipment.

FIG. 1 shows a schematic flow chart of an embodiment of a liver fibrosisrecognition method based on medical images according to the presentdisclosure.

As shown in FIG. 1 , the liver fibrosis recognition method based onmedical images applicable in a computing device can include thefollowing steps. According to different requirements, the order of thesteps in the flow chart may be changed, and some may be omitted.

In block 11, segmenting an interest region in each of a plurality ofliver medical images and obtaining a plurality of first binary images.

FIG. 2 is a schematic structural diagram of a network architecture fortraining a liver fibrosis recognition model. In some embodiments, a dataset comprised of a plurality of liver medical images and ground-truthlabels indicating liver fibrosis status can be constructed firstly, andthen the interest region in each of the plurality of liver medicalimages can be segmented to obtain a plurality of first binary images.For example, 232 liver patients are scanned up to 3 times each, usingeach time a different scanner type of image scanning device each time.The image scanning device can be, for example, an ultrasound device, aComputed Tomography (CT) device, a Magnetic Resonance Imaging (MRI)device, a positron emission tomography (PET) device, a single photonemission computed tomography (SPECT) device, a rotational angiographydevice, and other medical imaging investigative devices. In order tofacilitate understanding the present disclosure, the following describesan example of ultrasound images of liver patients.

Images of each liver patient are composed of up to 14 medical images anda total number of medical images of the data set is 6979. Among the 232liver patients, 95 (40.95%) patients have moderate to severe fibrosis,27 with severe liver steatosis, and all with hepatitis B. The data setcan be denoted as D={X_(i), y_(i)}, wherein i=1, . . . , 232, X_(i)={X¹,. . . , X^(K) ^(i) }, X_(i) represents K_(i) images of an i-th liverpatient, and y_(i) represents the ground-truth labels indicating liverfibrosis status of the i-th liver patient. y_(i) can be 1 or 0, wheny_(i) is 1, it indicates that the i-th liver patient has liver fibrosis,when y_(i) is 0, it indicates that the i-th liver patient does notsuffer from liver fibrosis.

In some embodiments, a pre-trained liver segmentation model can be usedto segment the interest region in each of the plurality of liver medicalimages to obtain a plurality of first binary images.

The interest region refers to a region in the liver medical images wherelesions can occur. An improved U-Net (deeply-supervised net) can be usedto detect the interest region in each of the plurality of liver medicalimages, and then segment or isolate the interest region from each of theplurality of liver medical images to obtain the plurality of firstbinary images. Pixel values corresponding to the interest region in thefirst binary image are 1, and pixel values corresponding to a regionthat is not of interest in the first binary image are 0.

The improved U-Net enables the semantics of the intermediate feature mapto be discernible at each image scale, resulting in better results thanwith an original U-Net. The improved U-Net is known in the prior art,and a process of segmenting the interest region using the improved.U-Net is also prior art. The present disclosure will not describe thesame in detail herein.

In block 12, creating a rectangular region for each first binary image,and obtaining a plurality of second binary images by generating a secondbinary image according to each rectangular region and the correspondingfirst binary image.

Based on a priori clinical practice, certain features are crucial forrecognition of liver fibrosis, e.g., a parenchyma texture and surfacenodularity. As FIG. 2 demonstrates, to make the liver fibrosisrecognition model focus on these certain features, a masking techniquecan be used. That is, obtaining a first binary image for each livermedical image, each first binary image can be a liver mask for eachliver medical image. Then, for each liver medical image, a rectangularregion for each first binary image can be created, and a second binaryimage can be generated according to each rectangular region and thecorresponding first binary image, ensuring that the second binary imagecaptures enough of the parenchyma texture and surface nodularity toextract meaningful features.

In some embodiments, the method of creating a rectangular region foreach first binary image includes: determining a center point, a highestpoint, and a left boundary and a right boundary of the interest regionin each first binary image; extending the highest point upward, by apreset number of pixels to obtain a target highest point; creating arectangular region according to the left boundary and the right boundaryof the interest region, wherein a midpoint of a lower edge of therectangular region is the center point, and a midpoint of an upper edgeof the rectangular region is the target highest point.

The rectangular region can be used to cover a top half of the liver mask(that is, to cover a top half of the interest region of the first binaryimage).

For example, the preset number of pixels can be 10. Extending thehighest point upward by the preset number of pixels to obtain a targethighest point and creating the rectangular region according to thetarget highest point, allows the rectangular region to cover an entireborder of the liver.

In some embodiments, the method of generating the second binary imageaccording to each rectangular region and the corresponding first binaryimage includes: updating pixel values corresponding to the rectangularregion in the first binary image to target pixel values; and generatingthe second binary image according to the updated pixel values in thefirst binary image.

Referring to FIG. 3 , FIG. 3(a). depicts a liver ultrasound image, whosefirst binary image is rendered. A white area in FIG. 3(b). correspondsto the liver mask, and FIG. 3(c). shows a generated second binary image,a white area in FIG. 3(c). is extended upward to cover the top border ofa liver.

In block 13, extracting a feature map of each liver medical image toobtain a plurality of feature maps.

Each liver medical image corresponds to one feature map. A plurality offeature maps can be obtained by extracting a feature map of each livermedical image.

In fact, ultrasound views of the liver can be roughly divided into 6categories, each of which focuses on different regions of the liver.

Referring to FIG. 4 , an example of liver Ultrasound images withdifferent image views is shown. From left to right: left hepatic lobe,midline abdomen view; left hepatic lobe, transverse abdomen view; righthepatic lobe, intercostal view 1; right hepatic lobe, intercostal view2; subcostal view 2; and subcostal view 2. Different manners ofview-specific analysis can push performance further.

In some embodiments, a dedicated deep CNN can be trained for each viewcategory to extract a feature map of each liver medical imagecorresponding to the view category. However, this drastically reduces atraining set for each dedicated deep CNN and increases six-fold thenumber of parameters, computation load, and memory consumption.

In some embodiments, the method of obtaining the plurality of featuremaps by extracting a feature map of each liver medical images includes:recognizing an image view of each liver medical image; generating aview-specific feature extracting model according to the image view ofeach liver medical image; and extracting a feature map of each livermedical image to obtain a plurality of feature maps using the generatedview-specific feature extracting model.

A pre-trained image view recognition model can be used to recognize theimage view of each liver medical image. A process of training the imageview recognition model can include: obtaining a sample data set, thesample data set including a plurality of liver medical image samples,and each liver medical image sample corresponds to one image view;obtaining a training set from the sample data set; calculating a lossvalue of a risk loss function by inputting the training set into aconvolutional neural network; when it is determined that the loss valueof the risk loss function reaches a convergence state, updatingparameters in an initial network framework of the convolutional neuralnetwork according to the risk loss value; and determining the initialnetwork framework after updating the parameters as an image viewrecognition model.

In some embodiments, the sample data set can be obtained by scanningsame part of a plurality of patients using various image scanningdevices for training the convolutional neural network to obtain an imageview recognition model.

Using a first ratio (e.g., 67.5%) dividing the plurality of livermedical image samples into the training set, using a second ratio (e.g.,12.5%) to divide the remaining plurality of liver medical image samplesinto a validation set, and using another ratio (e.g., 20%) for divisioninto a test set. Wherein a sum of the first ratio, the second ratio andthe third ratio is 1. Inputting the training set into an initial networkframework of the convolutional neural network, parameters in the initialnetwork framework need to be trained to be determined. The initialnetwork framework of the convolutional neural network includes: aplurality of convolution layers, a plurality of pooling layers, onecompression excitation layer SE, one global average merge layer, onefully connected layer, and one SoftMax output layer. The loss functionin the SoftMax output layer can be a cross entropy (CE) loss function oran aggregated cross entropy (ACE) loss function. After training theconvolutional neural network model, the verification set can be used tooptimize parameters in the convolutional neural network model, and thetest set can be used to test the performance of the convolutional neuralnetwork model.

A concept of the “style” parameters can be adapted to implement aview-specific parameterization (VSP) appropriate for ultrasound-basedfibrosis recognition. Such parameters refer to the affine normalizationparameters used in batch- or instance-normalization. If these parametersare switched out, keeping all other parameters constant, one can alterthe behavior of the CNN dramatically. Retaining view-specificnormalization parameters allows for the majority of parameters andprocessing to be shared across views. VSP is then realized with aminimal number of additional parameters.

Since the Ultrasound views of the liver are roughly divided into 6categories, 6 sets of normalization parameters can be created and bedenoted as Ø={ω₁, ω₂, ω₃, ω₄, ω₅, ω₆}. Each image view corresponds toone ω, and different image views correspond to different ω. A fullyconnected neural network includes a plurality of normalization layers,and each normalization layer includes two parameters, so there are atotal of 12 parameters.

After recognizing the image view of each liver medical image, the viewspecific feature extracting model can be generated according to theimage view of each liver medical image; the plurality of feature mapscan be obtained by extracting a feature map of each of the liver medicalimages using the generated view specific feature extracting model.

A truncated version of ResNet (only a first three layer blocks) can beused. This keeps enough spatial resolution prior to the liver mask. Thistruncated backbone is called “ResNet-3”.

In block 14, generating a plurality of input images according to theplurality of second binary images and corresponding to the plurality offeature maps.

Each input image can be obtained by calculating a cross product of thecorresponding second binary image and the corresponding feature map. Itis found that including the zeroed-out regions within the global averagepooling benefits performance. Their inclusion helps the implicit captureof liver size characteristics, which is another clinical ultrasoundmarker for liver fibrosis.

In block 15, iteratively training a liver fibrosis recognition modelbased on the plurality of input images.

A challenge with liver fibrosis studies is the variable number of livermedical images, each of which has a potentially different view. Ideally,all available liver medical images contribute to the final prediction.In the existing technology, a conventional approach is to aggregateindividual image-view predictions, e.g., taking a median. Thisconventional approach has drawbacks, as it is accomplished via a latefusion of independent and image-specific predictions. But, this does notallow the CNN to integrate the combined features across ultrasoundimages. A better approach fuses these features. The challenge is toallow for an arbitrary number of Ultrasound images in order to ensureflexibility and practicality.

The HeMIS approach to segmentation offers a promising strategy thatfuses features from arbitrary numbers of images using their first- andsecond-order moments. However, HeMIS fuses convolutional features earlyin its FCN pipeline, which is possible because it assumes pixel-to-pixelcorrespondence across images. ultrasound images violate this assumption.Instead, only global Ultrasound features can sensibly be fused together,which is accomplished through global hetero-image fusion (GHIF). It usesA={A^(k)} and M={M^(k)} to denote the set of FCN features and clinicalinterests region, respectively, for each liver medical image. Then GHIFaccepts any arbitrary set of FCN features to produce a study-wiseprediction:ŷ=f(g(A;M);w),g(A;M)=concat(mean(G), var(G), mean(G)),G={GAP(M ^(k) ⊙A ^(k))}_(k=1) ^(K).

While GHIF can effectively integrate arbitrary numbers of Ultrasoundimages within a study, it uses the same FCN feature extractor across allimages, treating them all identically. Yet, there are certain ultrasoundfeatures, such as vascular markers, that are specific to particularviews.

In some embodiments, the method of iteratively training a liver fibrosisrecognition model based on the plurality of input images includes:selecting a training data set from the plurality of input images; ineach iteration of training, randomly selecting K input images from thetraining data set of the same liver patient; fusing the K input imagesto obtain a fused image; training a fully connected neural network basedon a plurality of the fused images; acquiring predicted labels output bya fully connected layer of the fully connected neural network;calculating a prediction accuracy rate according to the ground-truthlabels indicating liver fibrosis status and the predicted labels;determining whether the prediction accuracy rate is greater than apreset accuracy rate threshold; when the prediction accuracy ratebecomes greater than the preset accuracy rate threshold, stopping thetraining of the fully connected neural network, to obtain the liverfibrosis recognition model.

Here, the GHIF also incorporates an operator, in addition to the first-and second-order moments, as that has a powerful hetero-fusion functionwhich can enhance a performance of the liver fibrosis recognition model.GHIF is a new but effective process applied to global feature vectors.Rather than always inputting all liver medical images when training, animportant strategy is choosing random combinations of the K livermedical images for every epoch. This provides a form of dataaugmentation and allows the CNN to learn from image signals that may besuppressed otherwise. Training with random combinations of images canmake GHIF's batch statistics unstable and for this reason, anormalization not relying on batch statistics, such asinstance-normalization, should be used.

In some embodiments, before training the liver fibrosis recognitionmodel, the method also includes: augmenting the plurality of inputimages by adjusting the plurality of input images with random changes ofbrightness, contrast, rotations, and scale. By so adjusting theplurality of input images, a number of the plurality of input images canbe expanded, thereby expanding a number of the training data set,thereby improving the generalization performance of the liver fibrosisrecognition model.

In block 16, obtaining a liver fibrosis recognition result by using theliver fibrosis recognition model to recognize a liver medical image tobe recognized.

The liver medical image to be recognized is inputted into the liverfibrosis recognition model. The liver fibrosis recognition model outputsa liver fibrosis recognition result after recognizing a liver medicalimage to be recognized. The liver fibrosis recognition result can beeither healthy or liver fibrosis.

First, to avoid a CNN overfitting on non-relevant image features(spurious or background features), the neural network is forced to focuson a clinical region of interest (ROI), encompassing the liverparenchyma and upper border. Second, by introducing global hetero-imagefusion (GHIF), the neural network is capable to fuse features from anyarbitrary number of images in an Ultrasound study, increasing itsversatility and flexibility. Finally, “style”-based view-specificparameterization (VSP) is applied to tailor the CNN processing based onthe particular view of each ultrasound image based on 6 common liverultrasound views, while keeping the majority of parameters the sameacross views. The result is a highly robust and practical liver fibrosisassessment solution. Experiments on a dataset of 610 Ultrasound patientstudies (6979 images) demonstrate that our pipeline contributes roughly7% and 22% improvements over conventional classifiers in partial areaunder the curve and recall at 90% precision respectively, validating ourapproach to this crucial problem.

FIG. 5 shows a schematic structural diagram of an embodiment of a liverfibrosis recognition device according to the present disclosure.

In some embodiments, the liver fibrosis recognition device 50 caninclude a plurality of function modules consisting of program codesegments. The program code of each program code segments in the devicefor the liver fibrosis recognition device 50 may be stored in a memoryof a computing device and executed by the at least one processor toperform (described in detail in FIG. 1 ) a function of processing amedical image.

In an embodiment, the liver fibrosis recognition device 50 can bedivided into a plurality of functional modules, according to theperformed functions. The functional module can include: a segmentationmodule 501, a creation module 502, an extraction module 503, ageneration module 504, a training module 505, and a recognition module506. A module as referred to in the present disclosure refers to aseries of computer program segments that can be executed by at least oneprocessor and that are capable of performing fixed functions, which arestored in a memory. In this embodiment, the functions of each modulewill be detailed in the following embodiments.

The segmentation module 501 is configured to segment an interest regionin each of a plurality of liver medical images and obtain a plurality offirst binary images.

FIG. 2 is a schematic structural diagram of a network architecture fortraining a liver fibrosis recognition model. In some embodiments, a dataset comprised of a plurality of liver medical images and ground-truthlabels indicating liver fibrosis status can be constructed firstly, andthen the interest region in each of the plurality of liver medicalimages can be segmented to obtain a plurality of first binary images.For example, 232 liver patients are scanned up to 3 times each, usingeach time a different scanner type of image scanning device each time.The image scanning device can be, for example, an ultrasound device, aComputed Tomography (CT) device, a Magnetic Resonance Imaging (MRI)device, a positron emission tomography (PET) device, a single photonemission computed tomography (SPECT) device, a rotational angiographydevice, and other medical imaging investigative devices. In order tofacilitate understanding the present disclosure, the following describesan example of ultrasound images of liver patients.

Images of each liver patient are composed of up to 14 medical images anda total number of medical images of the data set is 6979. Among the 232liver patients, 95 (40.95%) patients have moderate to severe fibrosis,27 with severe liver steatosis, and all with hepatitis B. The data setcan be denoted as D={X_(i), y_(j)}, wherein i=1, . . . , 232, X_(i)={X¹,. . . , X^(K) ^(i) }, X_(i) represents K_(i) images of an i-th liverpatient, and y_(i) represents the ground-truth labels indicating liverfibrosis status of the i-th liver patient. y_(i) can be 1 or 0, when y₁is 1, it indicates that the i-th liver patient has liver fibrosis, wheny_(i) is 0, it indicates that the i-th liver patient does not sufferfrom liver fibrosis.

In some embodiments, a pre-trained liver segmentation model can be usedto segment the interest region in each of the plurality of liver medicalimages to obtain a plurality of first binary images.

The interest region refers to a region in the liver medical images wherelesions can occur. An improved U-Net (deeply-supervised net) can be usedto detect the interest region in each of the plurality of liver medicalimages, and then segment or isolate the interest region from each of theplurality of liver medical images to obtain the plurality of firstbinary images. Pixel values corresponding to the interest region in thefirst binary image are 1, and pixel values corresponding to a regionthat is not of interest in the first binary image are 0.

The improved U-Net enables the semantics of the intermediate feature mapto be discernible at each image scale, resulting in better results thanwith an original U-Net. The improved U-Net is known in the prior art,and a process of segmenting the interest region using the improved U-Netis also prior art. The present disclosure will not describe the same indetail herein.

The creation module 502 is configured to create a rectangular region foreach first binary image, and obtain a plurality of second binary imagesby generating a second binary image according to each rectangular regionand the corresponding first binary image.

Based on a priori clinical practice, certain features are crucial forrecognition of liver fibrosis, e.g., a parenchyma texture and surfacenodularity. As FIG. 2 demonstrates, to make the liver fibrosisrecognition model focus on these certain features, a masking techniquecan be used. That is, obtaining a first binary image for each livermedical image, each first binary image can be a liver mask for eachliver medical image. Then, for each liver medical image, a rectangularregion for each first binary image can be created, and a second binaryimage can be generated according to each rectangular region and thecorresponding first binary image, ensuring that the second binary imagecaptures enough of the parenchyma texture and surface nodularity toextract meaningful features.

In some embodiments, the process of the creation module 502 creating arectangular region for each first binary image includes: determining acenter point, a highest point, and a left boundary and a right boundaryof the interest region in each first binary image; extending the highestpoint upward by a preset number of pixels to obtain a target highestpoint; creating a rectangular region according to the left boundary andthe right boundary of the interest region, wherein a midpoint of a loweredge of the rectangular region is the center point, and a midpoint of anupper edge of the rectangular region is the target highest point.

The rectangular region can be used to cover a top half of the liver mask(that is, to cover a top half of the interest region of the first binaryimage).

For example, the preset number of pixels can be 10. Extending thehighest point upward by the preset number of pixels to obtain a targethighest point and creating the rectangular region according to thetarget highest point, allows the rectangular region to cover an entireborder of the liver.

In some embodiments, the process of the creation module 502 generatingthe second binary image according to each rectangular region and thecorresponding first binary image includes: updating pixel valuescorresponding to the rectangular region in the first binary image totarget pixel values; and generating the second binary image according tothe updated pixel values in the first binary image.

Referring to FIG. 3 , FIG. 3(a). depicts a liver ultrasound image, whosefirst binary image is rendered. A white area in FIG. 3(b). correspondsto the liver mask, and FIG. 3(c). shows a generated second binary image,a white area in FIG. 3(c). is extended upward to cover the top border ofa liver.

the extraction module 503 is configured to extract a feature map of eachliver medical image to obtain a plurality of feature maps.

Each liver medical image corresponds to one feature map. A plurality offeature maps can be obtained by extracting a feature map of each livermedical image.

In fact, ultrasound views of the liver can be roughly divided into 6categories, which focus on different regions of the liver.

Referring to FIG. 4 , an example of liver Ultrasound images withdifferent image views is shown. From left to right: left hepatic lobe,midline abdomen view; left hepatic lobe, transverse abdomen view; righthepatic lobe, intercostal view 1; right hepatic lobe, intercostal view2; subcostal view 2; and subcostal view 2. Different manners ofview-specific analysis can help push performance further.

In some embodiments, a dedicated deep CNN can be trained for each viewcategory to extract a feature map of each liver medical imagecorresponding to the view category. However, this drastically reduces atraining set for each dedicated deep CNN and increases six-fold thenumber of parameters, computation, and memory consumption.

In some embodiments, in obtaining the plurality of feature maps byextracting a feature map of each liver medical image, the extractionmodule 503 can: recognize an image view of each liver medical image;generate a view-specific feature extracting model according to the imageview of each liver medical image; and extract a feature map of eachliver medical image to obtain a plurality of feature maps using thegenerated view-specific feature extracting model.

A pre-trained image view recognition model can be used to recognize theimage view of each liver medical image. A process of training the imageview recognition model can include: obtaining a sample data set, thesample data set including a plurality of liver medical image samples,and each liver medical image sample corresponds to one image view;obtaining a training set from the sample data set; calculating a lossvalue of a risk loss function by inputting the training set into aconvolutional neural network; when it is determined that the loss valueof the risk loss function reaches a convergence state, updatingparameters in an initial network framework of the convolutional neuralnetwork according to the risk loss value; and determining the initialnetwork framework after updating the parameters as an image viewrecognition model.

In some embodiments, the sample data set can be obtained by scanning asame part of a plurality of patients using various image scanningdevices for training the convolutional neural network to obtain an imageview recognition model.

Using a first ratio (e.g., 67.5%) to divide the plurality of livermedical image samples into the training set, using a second ratio (e.g.,12.5%) to divide the remaining plurality of liver medical image samplesinto a validation set, and using another ratio (e.g., 20%) for divisioninto a test set. Wherein a sum of the first ratio, the second ratio andthe third ratio is 1. Inputting the training set into an initial networkframework of the convolutional neural network, parameters in the initialnetwork framework need to be trained to be determined. The initialnetwork framework of the convolutional neural network includes: aplurality of convolution layers, a plurality of pooling layers, onecompression excitation layer SE, one global average merge layer, onefully connected layer, and one SoftMax output layer. The loss functionin the SoftMax output layer can be a cross entropy (CE) loss function oran aggregated cross entropy (ACE) loss function. After training theconvolutional neural network model, the verification set can be used tooptimize parameters in the convolutional neural network model, and thetest set can be used to test the performance of the convolutional neuralnetwork model.

A concept of the “style” parameters can be adapted to implement aview-specific parameterization (VSP) appropriate for ultrasound-basedfibrosis recognition. Such parameters refer to the affine normalizationparameters used in batch- or instance-normalization. If these parametersare switched out, keeping all other parameters constant, one can alterthe behavior of the CNN dramatically. Retaining view-specificnormalization parameters allows for the majority of parameters andprocessing to be shared across views. VSP is then realized with aminimal number of additional parameters.

Since the Ultrasound views of the liver are roughly divided into 6categories, 6 sets of normalization parameters can be created and bedenoted as Ø={ω1, ω2, ω3, ω4, ω5, ω6}. Each image view corresponds toone ω, and different image views correspond to different ω. A fullyconnected neural network includes a plurality of normalization layers,and each normalization layer includes two parameters, so, there are atotal of 12 parameters.

After recognizing the image view of each liver medical image, the viewspecific feature extracting model can be generated according to theimage view of each liver medical image; the plurality of feature mapscan be obtained by extracting a feature map of each of the liver medicalimages using the generated view specific feature extracting model.

A truncated version of ResNet (only a first three layer blocks) can beused. This keeps enough spatial resolution prior to the liver mask. Thistruncated backbone is called “ResNet-3”.

The generation module 504 is configured to generate a plurality of inputimages according to the plurality of second binary images andcorresponding to the plurality of feature maps.

Each input image can be obtained by calculating a cross product of thecorresponding second binary image and the corresponding feature map. Itis found that including the zeroed-out regions within the global averagepooling benefits performance. Their inclusion helps the implicit captureof liver size characteristics, which is another clinical ultrasoundmarker for liver fibrosis.

The training module 505 is configured to iteratively train liverfibrosis recognition model based on the plurality of input images.

A challenge with liver fibrosis studies is the variable number of livermedical images, each of which has a potentially different view. Ideally,all available liver medical images contribute to the final prediction.In the existing technology, a conventional approach is to aggregateindividual image-view predictions, e.g., taking a median. Thisconventional approach has drawbacks, as it is accomplished via a latefusion of independent and image-specific predictions. But, this does notallow the CNN to integrate the combined features across ultrasoundimages. A better approach fuses these features. The challenge is toallow for an arbitrary number of Ultrasound images in order to ensureflexibility and practicality.

The HeMIS approach to segmentation offers a promising strategy thatfuses features from arbitrary numbers of images using their first- andsecond-order moments. However, HeMIS fuses convolutional features earlyin its FCN pipeline, which is possible because it assumes pixel-to-pixelcorrespondence across images. ultrasound images violate this assumption.Instead, only global Ultrasound features can sensibly be fused together,which is accomplished through global hetero-image fusion (GHIF). It usesA={A^(k)} and M={M^(k)} to denote the set of FCN features and clinicalinterests region, respectively, for each liver medical image. Then GHIFaccepts any arbitrary set of FCN features to produce a study-wiseprediction:ŷ=f(g(A;M);w),g(A;M)=concat(mean(G), var(G), mean(G)),G={GAP(M ^(k) ⊙A ^(k))}_(k=1) ^(K).

While GHIF can effectively integrate arbitrary numbers of Ultrasoundimages within a study, it uses the same FCN feature extractor across allimages, treating them all identically. Yet, there are certain ultrasoundfeatures, such as vascular markers, that are specific to particularviews.

In some embodiments, the training module 505 iteratively training aliver fibrosis recognition model based on the plurality of input imagesincludes: selecting a training data set from the plurality of inputimages; in each iteration of training, randomly selecting K input imagesfrom the training data set of the same liver patient; fusing the K inputimages to obtain a fused image; training a fully connected neuralnetwork based on a plurality of the fused images; acquiring predictedlabels output by a fully connected layer of the fully connected neuralnetwork; calculating a prediction accuracy rate according to theground-truth labels indicating liver fibrosis status and the predictedlabels; determining whether the prediction accuracy rate is greater thana preset accuracy rate threshold; when the prediction accuracy ratebecomes greater than the preset accuracy rate threshold, stopping thetraining of the fully connected neural network, to obtain the liverfibrosis recognition model.

Here, the GHIF also incorporates an operator, in addition to the first-and second-order moments, as that has a powerful hetero-fusion functionwhich can enhance a performance of the liver fibrosis recognition model.GHIF is a new but effective process applied to global feature vectors.Rather than always inputting all liver medical images when training, animportant strategy is choosing random combinations of the K livermedical images for every epoch. This provides a form of dataaugmentation and allows the CNN to learn from image signals that may besuppressed otherwise. Training with random combinations of images canmake GHIF's batch statistics unstable and for this reason, anormalization not relying on batch statistics, such asinstance-normalization, should be used.

In some embodiments, before training the liver fibrosis recognitionmodel, the training module 505 also configured to: augment the pluralityof input images by adjusting the plurality of input images with randomchanges of brightness, contrast, rotations, and scale. By adjusting theplurality of input images, a number of the plurality of input images canbe expanded, thereby expanding a number of the training data set,thereby improving the generalization performance of the liver fibrosisrecognition model.

The recognition module 506 is configured to obtain a liver fibrosisrecognition result by using the liver fibrosis recognition model torecognize a liver medical image to be recognized.

The liver medical image to be recognized is inputted into the liverfibrosis recognition model. The liver fibrosis recognition model outputsa liver fibrosis recognition result after recognizing a liver medicalimage to be recognized. The liver fibrosis recognition result can behealthy or liver fibrosis.

First, to avoid, a CNN overfitting on non-relevant image features(spurious or background features), the neural network is forced to focuson a clinical region of interest (ROI), encompassing the liverparenchyma and upper border. Second, by introducing global hetero-imagefusion (GHIF), the neural network is capable to fuse features from anyarbitrary number of images in an Ultrasound study, increasing itsversatility and flexibility. Finally, “style”-based view-specificparameterization (VSP) is applied to tailor the CNN processing based onthe particular view of each ultrasound image based on 6 common liverultrasound views, while keeping the majority of parameters the sameacross views. The result is a highly robust and practical liver fibrosisassessment solution. Experiments on a dataset of 610 Ultrasound patientstudies (6979 images) demonstrate that our pipeline contributes roughly7% and 22% improvements over conventional classifiers in partial areaunder the curve and recall at 90% precision respectively, validating ourapproach to this crucial problem.

FIG. 6 shows a schematic structural diagram of a computing deviceaccording to an embodiment of the present disclosure.

As shown in FIG. 6 , the computing device 600 may include: at least onestorage device 601, at least one processor 602, at least onecommunication bus 603, and a transceiver 604.

It should be understood by those skilled in the art that the structureof the computing device 600 shown in FIG. 6 does not constitute alimitation of the embodiment of the present disclosure. The computingdevice 600 may be a bus type structure or a star type structure, and thecomputing device 600 may also include more or less hardware or softwarethan illustrated, or may have different component arrangements.

In at least one embodiment, the computing device 600 can include aterminal that is capable of automatically performing numericalcalculations and/or information processing in accordance with pre-set orstored instructions. The hardware of the terminal can include, but isnot limited to, a microprocessor, an application specific integratedcircuit, programmable gate arrays, digital processors, and embeddeddevices. The computing device 600 may further include an electronicdevice. The electronic device can interact with a user through akeyboard, a mouse, a remote controller, a touch panel or a voice controldevice, for example, an individual computers, tablets, smartphones,digital cameras, etc.

It should be noted that the computing device 600 is merely an example,and other existing or future electronic products may be included in thescope of the present disclosure, and are included in the reference.

In some embodiments, the storage device 601 can be used to store programcodes of computer readable programs and various data, such as the devicefor automatically delineating a clinical target volume of esophagealcancer 30 installed in the computing device 600, and automaticallyaccess to the programs or data with high speed during running of thecomputing device 600. The storage device 601 can include a read-onlymemory (ROM), a programmable read-only memory (PROM), an erasableprogrammable read only memory (EPROM), an one-time programmableread-only memory (OTPROM), an electronically-erasable programmableread-only memory (EEPROM), a compact disc read-only memory (CD-ROM), orother optical disk storage, magnetic disk storage, magnetic tapestorage, or any other non-transitory storage medium readable by thecomputing device 600 that can be used to carry or store data.

In some embodiments, the at least one processor 602 may be composed ofan integrated circuit, for example, may be composed of a single packagedintegrated circuit, or may be composed of a plurality of integratedcircuits of same function or different functions. The at least oneprocessor 602 can include one or more central processing units (CPU), amicroprocessor, a digital processing chip, a graphics processor, andvarious control chips. The at least one processor 602 is a control unitof the computing device 600, which connects various components of thecomputing device 600 using various interfaces and lines. By running orexecuting a computer program or modules stored in the storage device601, and by invoking the data stored in the storage device 601, the atleast one processor 602 can perform various functions of the computingdevice 600 and process data of the computing device 600.

In some embodiments, the least one bus 603 is used to achievecommunication between the storage device 601 and the at least oneprocessor 602, and other components of the computing device 600.

Although it is not shown, the computing device 600 may further include apower supply (such as a battery) for powering various components. Insome embodiments, the power supply may be logically connected to the atleast one processor 602 through a power management device, thereby, thepower management device manages functions such as charging, discharging,and power management. The power supply may include one or more a DC orAC power source, a recharging device, a power failure detection circuit,a power converter or inverter, a power status indicator, and the like.The computing device 600 may further include various sensors, such as aBLUETOOTH module, a Wi-Fi module, and the like, and details are notdescribed herein.

It should be understood that the described embodiments are forillustrative purposes only and are not limited by the scope of thepresent disclosure.

The above-described integrated unit implemented in a form of softwarefunction modules can be stored in a computer readable storage medium.The above software function modules are stored in a storage medium, andincludes a plurality of instructions for causing a computing device(which may be a personal computer, or a network device, etc.) or aprocessor to execute the method according to various embodiments of thepresent disclosure.

In a further embodiment, in conjunction with FIG. 1 , the at least oneprocessor 602 can execute an operating device and various types ofapplications (such as the liver fibrosis recognition device 50)installed in the computing device 600, program codes, and the like. Forexample, the at least one processor 602 can execute the modules 501-505.

In at least one embodiment, the storage device 601 stores program codes.The at least one processor 602 can invoke the program codes stored inthe storage device 601 to perform related functions. For example, themodules described in FIG. 5 are program codes stored in the storagedevice 601 and executed by the at least one processor 602, to implementthe functions of the various modules.

In at least one embodiment, the storage device 601 stores a plurality ofinstructions that are executed by the at least one processor 602 toimplement all or part of the steps of the method described in theembodiments of the present disclosure.

Specifically, the storage device 601 stores the plurality ofinstructions which when executed by the at least one processor 602,causes the at least one processor 602 to: segment an interest region ineach of a plurality of liver medical images and obtain a plurality offirst binary images; create a rectangular region for each first binaryimage, and obtain a plurality of second binary images by generating asecond binary image according to each rectangular region and thecorresponding first binary image; extract a feature map of each livermedical image to obtain a plurality of feature maps; generate aplurality of input images according to the plurality of second binaryimages and corresponding to the plurality of feature maps; iterativelytrain a liver fibrosis recognition model based on the plurality of inputimages; and obtain a liver fibrosis recognition result by using theliver fibrosis recognition model to recognize a liver medical image tobe recognized.

The embodiment of the present disclosure further provides a computerstorage medium, and the computer storage medium store a program thatperforms all or part of the steps including any of the method describedin the above embodiments.

A non-transitory storage medium having stored thereon instructions that,when executed by a processor of a computing device, causes the computingdevice to perform an liver fibrosis recognition method, the methodcomprising: segmenting an interest region in each of a plurality ofliver medical images and obtaining, a plurality of first binary images;creating a rectangular region for each first binary image, and obtaininga plurality of second binary images by generating a second binary imageaccording to each rectangular region and the corresponding first binaryimage; extracting a feature map of each liver medical image to obtain aplurality of feature maps; generating a plurality of input imagesaccording to the plurality of second binary images and corresponding tothe plurality of feature maps; iteratively training a liver fibrosisrecognition model based on the plurality of input images; and obtaininga liver fibrosis recognition result by using the liver fibrosisrecognition model to recognize a liver medical image to be recognized.

It should be noted that, for a simple description, the above methodembodiments expressed as a series of action combinations, but thoseskilled in the art should understand that the present disclosure is notlimited by the described action sequence. According to the presentdisclosure, some steps in the above embodiments can be performed inother sequences or simultaneously. Secondly, those skilled in the artshould also understand that the embodiments described in thespecification are all optional embodiments, and the actions and unitsinvolved are not necessarily required by the present disclosure.

In the above embodiments, descriptions of each embodiment has differentfocuses, and when there is no detail part in a certain embodiment,please refer to relevant parts of other embodiments.

In several embodiments provided in the preset application, it should beunderstood that the disclosed apparatus can be implemented in otherways. For example, the device embodiments described above are merelyillustrative. For example, divisions of the unit are only a logicalfunction division, and there can be other division ways in actualimplementation.

The modules described as separate components may or may not bephysically separated, and the components displayed as modules may or maynot be physical units. That is, it can locate in one place, ordistribute to a plurality of network units. Some or all of the modulescan be selected according to actual needs to achieve the purpose of thesolution of above embodiments.

In addition, each functional unit in each embodiment of the presentdisclosure can be integrated into one processing unit, or can bephysically present separately in each unit, or two or more units can beintegrated into one unit. The above integrated unit can be implementedin a form of hardware or in a form of a software functional unit.

It is apparent to those skilled in the art that the present disclosureis not limited to the details of the above-described exemplaryembodiments, and the present disclosure can be embodied in otherspecific forms without departing from the spirit or essentialcharacteristics of the present disclosure. Therefore, the presentembodiments are to be considered as illustrative and not restrictive,and the scope of the present disclosure is defined by the appendedclaims instead all changes in the meaning and scope of equivalentelements are included in the present disclosure. Any reference signs inthe claims should not be construed as limiting the claim.

The above embodiments are only used to illustrate technical solutions ofthe present disclosure, rather than restrictions on the technicalsolutions. Although the present disclosure has been described in detailwith reference to the above embodiments, those skilled in the art shouldunderstand that the technical solutions described in one embodiments canbe modified, or some of technical features can be equivalentlysubstituted, and these modifications or substitutions do not detractfrom the essence of the corresponding technical solutions from the scopeof the technical solutions of the embodiments of the present disclosure.

What is claimed is:
 1. A liver fibrosis recognition method based onmedical images applicable in a computing device, the method comprising:segmenting an interest region in each of a plurality of liver medicalimages and obtaining a plurality of first binary images; creating arectangular region for each first binary image, and obtaining aplurality of second binary images by generating a second binary imageaccording to each rectangular region and the corresponding first binaryimage; extracting a feature map of each liver medical image to obtain aplurality of feature maps; generating a plurality of input imagesaccording to the plurality of second binary images and corresponding tothe plurality of feature maps; iteratively training a liver fibrosisrecognition model based on the plurality of input images; and obtaininga liver fibrosis recognition result by using the liver fibrosisrecognition model to recognize a liver medical image to be recognized.2. The liver fibrosis recognition method of claim 1, the method ofcreating a rectangular region for each first binary image comprising:determining a center point, a highest point, and a left boundary and aright boundary of the interest region in each first binary image;extending the highest point upward by a preset number of pixels toobtain a target highest point; creating a rectangular region accordingto the left boundary and the right boundary of the interest region,wherein a midpoint of a lower edge of the rectangular region is thecenter point, and a midpoint of an upper edge of the rectangular regionis the target highest point.
 3. The liver fibrosis recognition method ofclaim 2, the method of generating the second binary image according toeach rectangular region and the corresponding first binary imagecomprising: updating pixel values corresponding to the rectangularregion in the first binary image to a target pixel values; andgenerating the second binary image according to the updated pixel valuesin the first binary image.
 4. The liver fibrosis recognition method ofclaim 3, the method of obtaining the plurality of feature maps byextracting a feature map of each liver medical images comprising:recognizing an image view of each liver medical image; generating a viewspecific feature extracting model according to the image view of eachliver medical image; and extracting a feature map of each liver medicalimage to obtain a plurality of feature maps using the generated viewspecific feature extracting model.
 5. The liver fibrosis recognitionmethod of claim 4, the method of iteratively training a liver fibrosisrecognition model based on the plurality of input images comprising:selecting a training data set from the plurality of input images; ineach iteration of training, randomly selecting K input images from thetraining data set of the same liver patient; fusing the K input imagesto obtain a fused image; training a fully connected neural network basedon a plurality of the fused images; acquiring predicted labels output bya fully connected layer of the fully connected neural network;calculating a prediction accuracy rate according to the ground-truthlabels indicating liver fibrosis status and the predicted labels;determining whether the prediction accuracy rate is greater than apreset accuracy rate threshold; when the prediction accuracy rate isgreater than the preset accuracy rate threshold, stopping the trainingof the fully connected neural network, to obtain the liver fibrosisrecognition model.
 6. The liver fibrosis recognition method of claim 5,before training the liver fibrosis recognition model, furthercomprising: augmenting the plurality of input images by adjusting theplurality of input images with random changes of brightness, contrast,rotations, and scale.
 7. The liver fibrosis recognition method of claim5, each of the plurality of input images being generated by calculatinga cross product of the corresponding second binary image and thecorresponding feature map.
 8. A computing device, comprising: at leastone processor; and a storage device storing one or more programs whichwhen executed by the at least one processor, causes the at least oneprocessor to: segment an interest region in each of a plurality of livermedical images and obtain a plurality of first binary images; create arectangular region for each first binary image, and obtain a pluralityof second binary images by generating a second binary image according toeach rectangular region and the corresponding first binary image;extract a feature map of each liver medical image to obtain a pluralityof feature maps; generate a plurality of input images according to theplurality of second binary images and corresponding to the plurality offeature maps; iteratively train a liver fibrosis recognition model basedon the plurality of input images; obtain a liver fibrosis recognitionresult by using the liver fibrosis recognition model to recognize aliver medical image to be recognized.
 9. The computing device of claim8, wherein the at least one processor creating a rectangular region foreach first binary image comprises: determining a center point, a highestpoint, and a left boundary and a right boundary of the interest regionin each first binary image; extending the highest point upward by apreset number of pixels to obtain a target highest point; creating arectangular region according to the left boundary and the right boundaryof the interest region, wherein a midpoint of a lower edge of therectangular region is the center point, and a midpoint of an upper edgeof the rectangular region is the target highest point.
 10. The computingdevice of claim 9, wherein the at least one processor generating thesecond binary image according to each rectangular region and thecorresponding first binary image comprises: updating pixel valuescorresponding to the rectangular region in the first binary image to atarget pixel values; and generating the second binary image according tothe updated pixel values in the first binary image.
 11. The computingdevice of claim 10, wherein the at least one processor obtaining theplurality of feature maps by extracting a feature map of each livermedical images comprises: recognizing an image view of each livermedical image; generating a view specific feature extracting modelaccording to the image view of each liver medical image; and extractinga feature map of each liver medical image to obtain a plurality offeature maps using the generated view specific feature extracting model.12. The computing device of claim 11, wherein the at least one processoriteratively training a liver fibrosis recognition model based on theplurality of input images comprises: selecting a training data set fromthe plurality of input images; in each iteration of training, randomlyselecting K input images from the training data set of the same liverpatient; fusing the K input images to obtain a fused image; training afully connected neural network based on a plurality of the fused images;acquiring predicted labels output by a fully connected layer of thefully connected neural network; calculating a prediction accuracy rateaccording to the ground-truth labels indicating liver fibrosis statusand the predicted labels; determining whether the prediction accuracyrate is greater than a preset accuracy rate threshold; when theprediction accuracy rate is greater than the preset accuracy ratethreshold, stopping the training of the fully connected neural network,to obtain the liver fibrosis recognition model.
 13. The computing deviceof claim 12, before training the liver fibrosis recognition model, theat least one processor is further caused to: augment the plurality ofinput images by adjusting the plurality of input images with randomchanges of brightness, contrast, rotations, and scale.
 14. The computingdevice of claim 13, each of the plurality of input images beinggenerated by calculating a cross product of the corresponding secondbinary image and the corresponding feature map.
 15. A non-transitorystorage medium having stored thereon instructions that, when executed bya processor of a computing device, causes the computing device toperform a liver fibrosis recognition method based on medical images, themethod comprising: segmenting an interest region in each of a pluralityof liver medical images and obtaining a plurality of first binaryimages; creating a rectangular region for each first binary image, andobtaining a plurality of second binary images by generating a secondbinary image according to each rectangular region and the correspondingfirst binary image; extracting a feature map of each liver medical imageto obtain a plurality of feature maps; generating a plurality of inputimages according to the plurality of second binary images andcorresponding to the plurality of feature maps; iteratively training aliver fibrosis recognition model based on the plurality of input images,obtaining a liver fibrosis recognition result by using the liverfibrosis recognition model to recognize a liver medical image to berecognized.
 16. The non-transitory storage medium of claim 15, themethod of creating a rectangular region for each first binary imagecomprising: determining a center point, a highest point, and a leftboundary and a right boundary of the interest region in each firstbinary image; extending the highest point upward by a preset number ofpixels to obtain a target highest point; creating a rectangular regionaccording to the left boundary and the right boundary of the interestregion, wherein a midpoint of a lower edge of the rectangular region isthe center point, and a midpoint of an upper edge of the rectangularregion is the target highest point.
 17. The non-transitory storagemedium of claim 16, the method of generating the second binary imageaccording to each rectangular region and the corresponding first binaryimage comprising: updating pixel values corresponding to the rectangularregion in the first binary image to a target pixel values; andgenerating the second binary image according to the updated pixel valuesin the first binary image.
 18. The non-transitory storage medium ofclaim 17, the method of obtaining the plurality of feature maps byextracting a feature map of each liver medical images comprising:recognizing an image view of each liver medical image; generating a viewspecific feature extracting model according to the image view of eachliver medical image; and extracting a feature map of each liver medicalimage to obtain a plurality of feature maps using the generated viewspecific feature extracting model.
 19. The non-transitory storage mediumof claim 18, the method of iteratively training a liver fibrosisrecognition model based on the plurality of input images comprising:selecting a training data set from the plurality of input images; ineach iteration of training, randomly selecting K input images from thetraining data set of the same liver patient; fusing the K input imagesto obtain a fused image; training a fully connected neural network basedon a plurality of the fused images; acquiring predicted labels output bya fully connected layer of the fully connected neural network;calculating a prediction accuracy rate according to the ground-truthlabels indicating liver fibrosis status and the predicted labels;determining whether the prediction accuracy rate is greater than apreset accuracy rate threshold; when the prediction accuracy rate isgreater than the preset accuracy rate threshold, stopping the trainingof the fully connected neural network, to obtain the liver fibrosisrecognition model.
 20. The non-transitory storage medium of claim 19,before training the liver fibrosis recognition model, the method furthercomprising: augmenting the plurality of input images by adjusting theplurality of input images with random changes of brightness, contrast,rotations, and scale.